Health technology assessment and evidence-based medicine: what are we talking about?

This Value in Health Special Issue describes technology assessment (TA) programs in five countries. The primary purpose is to provide information about the ways coverage and related decisions are made in each country, and how TA is used to support those decisions. For this purpose each article stands on its own. Individually they are helpful to understand the decision-making processes, as well as the methods used to assess technology in these countries. But when the articles are considered together, they tell us much more. One immediate observation is the extent to which TA is being used. These articles describe a very impressive range of programs. These are not intellectual exercises; they are integral parts of national decision-making processes. TA has clearly come a long way from its origins about 30 years ago. A second observation relates to the diversity of the programs. Given that all the programs stem from a common source; this is more surprising. They all have the same motivation, which is to understand the effects of health-care technologies on important outcomes. And they all share a commitment to the same approach, which is that to the greatest extent possible the understanding should be based on rigorous evidence and analytical methods. From this, one might expect the programs to be very similar, differing only in details dictated by local conditions such as committee structures. But the differences we see are considerably deeper than just committee structures; they affect such things as the types of decisions to which TA is applied, the mandates given to those applying them, the types of outcomes they should consider, the criteria for drawing conclusions, and the extent to which the conclusions are binding. Looking across the spectrum of programs one might wonder whether the authors are really talking about the same thing. Some emphasize an analysis of evidence and for all intents and purposes stop there. Some look more like health economics and outcomes research. And for others the centerpiece is a cost-effectiveness analysis. Just what is TA, if it can morph into so many different forms? Are the programs actually doing very different things—evidence-based medicine (EBM), outcomes analysis, costeffectiveness analysis—and are we being polite to call them all TA? Does the interpretation of the term “technology assessment” vary so widely that in fact there is no such thing? Or does TA encompass all of these as well as other disciplines, and the different programs just vary in the extent to which they use them? Health TA as defined in International Society For Pharmacoeconomics and Outcomes Research’s (ISPOR) Health-Care Cost, Quality, and Outcomes: ISPOR Book Terms [1] is “a form of policy research that examines shortand long-term consequences of the application of a health-care technology. Properties assessed include evidence of safety, efficacy, patient-reported outcomes, real world effectiveness, cost and cost-effectiveness as well as social, legal, ethical, and political impacts” [1]. The breadth of this definition clearly argues for the third interpretation in the previous paragraph—that TA is an umbrella term that encompasses a wide variety of applications and methods. But if we accept this, then what are we to make of the fact that some of the programs described in this Special Issue only do, for example, an evaluation of evidence for safety and efficacy? Some of the programs not only fail to perform a cost-effectiveness analysis; they are actually forbidden to do so. Should we cast these programs out of TA, and say that they just practice EBM? The most obvious resolution to these questions, I believe, is to recognize that a “full” TA has several different parts, and that different programs emphasize different parts. But the programs described in this Special Issue illustrate something more interesting than this. There is an order to the parts methodologically, making it reasonable to consider them “stages” towards a full TA, and the different programs have progressed to different stages. Thus some programs might be considered not just “different,” in some arbitrary sense, but “further along” toward a full TA. The four main stages are apparent in ISPOR’s definition. The first is an evidence analysis—a systematic evaluation of evidence for a technology and a requirement of good evidence for such things as coverage, placement on formularies, and affirmative guidelines. This stage corresponds to the evidence-based guidelines (EBGs) part of EBM. The second stage is an outcomes analysis. In this stage there is an estimation of the magnitude of the effects of the technology on the desired clinical outcomes (the “benefits”) and on potential harms such as side effects and risks (the “risks”). This stage also includes a comparison of benefits and risks, to determine if the “benefit–risk ratio” is sufficiently high to justify the technology. The third and fourth stages are analyses of costs and cost-effectiveness; here the researcher estimates the effect of the technology on costs and compares the clinical effects against the costs to determine if the ratio is sufficiently high. The last stage is the analysis of the ethical and legal implications of the technology. The methodological progression of these stages is apparent. One can not estimate the magnitude of clinical outcomes (stage 2) without first evaluating the clinical evidence (stage 1). One can not compare the costs and cost-effectiveness (stages 3 and 4) without estimating the effects on the clinical outcomes (stage 2). And one can not think about ethical and legal implications of the technology (stage 5), until one knows something about the costs and cost-effectiveness (stages 3 and 4). Address correspondence to: David Eddy, Archimedes, Inc., 201 Mission Street Fl 29, San Francisco, CA 94105-1831, USA. E-mail: [email protected] 10.1111/j.1524-4733.2009.00551.x Volume 12 • Supplement 2 • 2009 V A L U E I N H E A L T H